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BRIEFING NOTE 


ANALYSING AND COMPARING 


VET QUALIFICATIONS 


VET qualifications must be relevant at national and local levels 
while opening the door to international comparability 


Pressure on European education and training sys- 
tems has increased in recent years, with technolog- 
ical and demographic trends reshaping demand for 
skills and qualifications and making lifelong learning 
a necessity, for both individuals and countries. Vo- 
cational education and training (VET) provision has 
to respond to rapidly changing labour market needs 
and to match qualification supply to skills demand. 
Improving its quality, relevance and agility is, there- 
fore, at the heart of the European VET policy frame- 
work 2021-25, which emphasises the importance of 
international cooperation, learner and worker mobility 
and recognition of learning outcomes (’). 

To support the implementation of policies strength- 
ening cross-country transparency and comparability 
of qualifications, Cedefop has conducted a study into 
methods for analysing and comparing the profile and 
content of VET qualifications; these still largely differ 
between European countries (?). This briefing note 
outlines the two main objectives addressed, as well 
as the solutions identified and their implications for 
researchers and policy-makers. 


OBJECTIVES 

1. Better comparison of VET qualifications 
Traditionally, VET qualifications are developed by 
country authorities addressing mainly their own na- 
tional and regional needs. This allows for direct dia- 


(! 
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The current EU VET policy framework is essentially defined by 
the renewed European skills agenda, the (first ever) Council 
recommendation on vocational education and training, and the 
Osnabruck declaration, with which the ministers responsible 
for VET in the EU Member States, the EU candidate countries 
and the EEA countries, social partners and the European Com- 
mission agreed on a new set of policy actions to complement 
and operationalise the vision and strategic objectives formu- 
lated in the Council VET recommendation. 


Profile: structure and scope of a qualification; content: ana- 
lytical description of the different parts forming a full qualifica- 
tion. A study was conducted in 10 countries (AT, BG, DK, FI, 
FR, IE, LT, NL, ES, UK) to identify possible methods for, and 
practical applications of, the analysis and comparison of qual- 
ifications. The two occupation IT technicians and health care 
assistants served as examples. (Cedefop, 2021, forthcoming). 


RS 
Y~ 


BRIEFING NOTE | AUGUST 2021 | ISSN 1831-2411 


logue between the users and suppliers of qualifica- 
tions, ensuring overall relevance of training. In recent 
times, however, this approach has been challenged 
by sweeping technological changes and ever more 
globalised labour markets and supply chains. Skills 
and competences, while used locally, are increas- 
ingly shaped by global trends and the calls for wid- 
er comparability of the content of VET programmes 
and qualifications are multiplying (°). Cedefop’s study 
addresses the challenges posed by the tensions be- 
tween local needs and global demands, and opens 
possible ways forward to be discussed at political 
level. 


2. Better feedback between work and VET 

In the past few years, Cedefop has built up its skills 
intelligence capacity through the analysis of online 
job vacancies, offering the capacity to generate fast 
and detailed information on labour market trends and 
European companies’ skill needs as they unfold. At 
the same time, Cedefop has looked at the supply of 


°¢) As illustrated by skills competitons like EuroSkills and 
WorldSkills, a core of VET skills and competences is shared 
and can indeed be compared internationally. Cedefop’s meth- 
odological work in this field is carried out in close dialogue with 
WorldSkills. 
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knowledge, skills and competences, forecasting like- 
ly future developments of VET systems as they face 
new challenges and demands. 

Good VET governance and quality assurance re- 
quire strong feedback mechanisms between VET 
providers and labour market stakeholders. While a lot 
has been done to optimise the match between VET 
needs and supply, there is room for more targeted 
and ‘granular’ feedback at the level of single qualifi- 
cations/programmes. More systematic dialogue be- 
tween VET providers and labour market actors in this 
field can help ensure that: 

«= the learning outcomes expressed in curricula 
closely reflect labour market needs; 

= the intentions of VET providers have been translat- 
ed into traceable individual VET graduate skills; 

= experiences at the workplace with the graduates 
are being fed back to VET providers. 


Cedefop’s study on the analysis and comparison 
of VET qualifications (see footnote 2) directly supports 
this feedback loop: it proposes methods for collecting 
the views of labour market actors on qualifications’ 
fitness for purpose, promoting deeper understanding 
of the relationship between the learning outcomes in- 
tended by VET systems and actual outcomes as ex- 
perienced in labour markets. 


BOX 1. LEARNING OUTCOMES IN QUALIFICATION 
DESCRIPTIONS 


Learning outcomes enable a breakdown of qualifi- 
cations that can be applied in different national con- 
texts. They offer stakeholders from different countries 
a common platform for the review and renewal of 
qualifications. As they sharpen our understanding of 
the content of qualifications, they allow us to calibrate 
and orient them in such a way that they open access, 
within and across borders, to labour markets and fur- 
ther learning, allowing people to ‘take their qualifica- 
tions with them’. 

Analysis and comparison of learning outcomes dis- 
tinguish between intended and acquired outcomes: 
they allow the objectives set by the developers of a 
qualification to be compared with what a person is re- 
ally able to do, as observed, say, by his/her employer. 
The intentions expressed in curricula will always have 
to be realised through learning and eventually test- 
ed in real work situations. Tracking gaps between 
intended and acquired learning outcomes is crucial 
for better matching qualifications to real labour mar- 
ket needs. This effort will have to be continuous, as 
needs change quickly, requiring swift responses. 


Source: Cedefop (forthcoming). 
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FIGURE 1. THE FEEDBACK LOOP 


THE SHARED 
(JOINT) WRITING 
OF LEARNING 
OUTCOMES 


INTENTIONS INTO 
STANDARDS, 
Ses PROGRAMMES OR 
i CURRICULA 


ACTUAL LEARNING 
LEARNING OUTCOMES 
OUTCOMES AS INTENTIONS AS 
EXPERIENCED INTERPRETED BY 
AND MONITORED TEACHERS AND 


BY EMPLOYERS TRAINERS 


INTENTIONS 


Source: Cedefop (forthcoming). 


GETTING TO THE BOTTOM OF 
VET QUALIFICATIONS 


Understanding the profile and content of qualifica- 
tions requires capturing their intentions and the way 
these are expressed through learning outcomes (). 
This approach supports both the comparison of qual- 
ifications for mutual learning and efforts to close the 
feedback loop. For both, an in-depth analysis of the 
learning outcomes forming the basis of qualifications 
is needed. 

The combined mapping of transversal skills listed 
for two occupations, IT technicians and healthcare 
assistants, carried out in 10 countries, makes it clear 
that countries’ qualifications share a fair amount of 
skills and competences. This proves that VET quali- 
fications from different countries address roughly the 
same skill needs. At the same time, the comparison 
revealed a considerable degree of variation in the way 
countries describe these transversal skills. While most 
emphasised in their descriptions learning outcomes 
such as ‘interact with others’ and ‘follow safety pre- 
cautions in work practices’, only a few listed ‘demon- 
strate curiosity’ or ‘foreign languages’. 

Countries also differed in how they address oc- 
cupation-specific skills. While some operate with de- 
tailed and lengthy lists of technical skills to be mas- 
tered, others use broader occupational and technical 
skills, this latter approach being based on the idea 
that technical change is inevitable and impossible to 
predict in detail. The comparison of the 10 countries 
conveys the need for, and the difficulty of, balancing 


(*) See Box 1. 
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occupational and transversal skills, and hints at how 

the different national approaches can support future 

developments. 
Cedefop concludes that progress is required in the 

following areas: 

= further development of reference systems sup- 
porting analysis and comparison within and be- 
tween countries at different levels; 

= promotion of a more systematic overall use of the 
learning outcomes approach, through a commonly 
agreed structure and extensive sharing of data; 

= countries’ agreement on a common format for 
using learning outcomes in qualification descrip- 
tions; 

= exploration of automated gathering of data, mak- 
ing it possible to scale up analysis and compari- 
son. 


Some of these requirements are discussed below. 


IDENTIFYING REFERENCE 
POINTS 


To compare learning outcomes included in national 
qualifications and identify their similarities and dif- 
ferences, countries need to be enabled to map them 
against a common reference point. The Cedefop 
study looked into various existing reference systems 
and tested their strengths and limitations, again on the 
basis of the two occupations in the 10 countries. The 
aim was to identify the reference system best suited 
to capturing the overall profile of qualifications while 
providing detailed insights into the different types of 
skills covered: occupation-specific versus transver- 
sal skills. The following systems were considered: 
= ESCO, the multilingual European classification of 
skills, competences, qualifications and occupa- 
tions (version 1); 
= O*Net, the USA’s Occupational information net- 
work and primary source of vocational intelligence; 
= WorldSkills standards specifications (WSSS), 
which are used as the reference point for the 
WorldSkills competition; 
= the VQTS model (Vocational qualification transfer 
system), competence matrices developed and ap- 
plied in a series of EU-funded projects. 


The comparison of the two occupations in 10 
countries revealed that ESCO is currently best suited 
for analysing and comparing VET qualifications and 
can serve as a ‘translation hub’ for comparing quali- 
fications and identifying their match to labour market 
needs. ESCO is available in 27 languages and is thus 
in a unique position to serve comparisons of national 
qualifications. Distinguishing between transversal and 


BRIEFING NOTE | AUGUST 2021 | ISSN 1831-2411 


occupation-specific skills, it offers a conceptual gran- 
ularity allowing detailed analysis and comparison. 

Nevertheless, Cedefop has concluded that, for the 
moment, no single reference point can serve all pur- 
poses. While approaches like ESCO and O*NET offer 
a good starting point, they would need some adjust- 
ments to tackle all required tasks (°). Other reference 
systems such as WSSS, can complement them where 
needed. In any case, a reference point should act as 
a terminological ‘translation hub’ only and not take on 
a normative function or be used as a standard. To be 
followed up... 


NATIONAL VET QUALIFICATION 
SOURCES 


National qualifications are increasingly described on 
the basis of learning outcomes; progress has been 
remarkable over the past decade. Varying in struc- 
ture, length and detail, however, these descriptions 
are not always well suited to analysis and compari- 
son. 

The Cedefop study showed, for example, that for 
IT technicians, five of the countries where this occu- 
pation was examined (IE, ES, AT, Fl, UK) considered 
knowledge of ICT encryption services essential; an- 
other five (BG, DK, FR, LT, NL) had not included it 
in their qualification descriptions. Similarly, five coun- 
tries judged that IT technicians needed to be able to 
manage email hosting services autonomously (BG, IE, 
ES, AT, Fl), while another five did not mention this re- 
quirement (DK, FR, LT, NL, UK). 

While the lack of a common description format 
hinders analysis and comparison, the uneven devel- 
opment of national qualification databases prevents 
upscaling of methodologies. Hence access to, and 
comparability of, qualifications data are still limited 
owing to the use of incompatible data formats (°) and 
incomplete coverage. 

If addressed, these weaknesses can be turned into 
strengths and allow for much more extensive analysis 
and comparison of qualifications throughout Europe. 
For methodologies to be upscaled, thus becoming 
available to a wider group of stakeholders, agreement 
on a common format for describing qualifications is 
indispensable: it would increase the overall transpar- 
ency of qualifications and make it easier for learners, 
employees and employers to grasp fully the content 


() The main strength of ESCO lies in its very detailed approach 
to skills and competences, covering all relevant European la- 
bour market sectors. O*Net was developed for the US labour 
market. Its main strength lies in its frequent regular updates, 
ensuring a high degree of relevance to users. 

(°) The 2017 update of the EQF recommendation includes an an- 
nex (VI) on how to share data on qualifications across Europe. 
The continued implementation of this will directy support pro- 
gress in this area. 
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and profile of qualifications. Such a common format 
could be based on features of the Europass certificate 
supplement (7) and would underpin the automated 
analysis and comparison of qualifications (°). 


TOOLS 

1. Automated qualification analysis 

To be scalable in the future, the review and compari- 
son of qualifications needs to rely on digital tools, in- 
cluding artificial intelligence. A digital tool supporting 
the automated analysis and comparison of the learn- 
ing outcomes of VET qualifications would add value, 
but not all conditions are currently met to realise this 
approach. Incompatible data formats and significant 
differences in content structuring impede the use of 
automated processes. While machine learning (artifi- 
cial intelligence) could help overcome some of these 
problems, it would require a considerable up-front 
investment in terms of time and resources. But, giv- 
en the continued roll-out of national databases and a 
possible agreement on data formats and presenta- 
tion templates, there is significant potential for auto- 
mated analysis. 

A pilot project was launched in 2020 to test an 
automatic linking of qualifications data to ESCO. So 
far, experiences from this project seem to concur with 
Cedefop’s own tests and will eventually provide more 
insight into the conditions needed for automatic anal- 
ysis and comparison. 


2. Employer feedback 

Employer reflection surveys asking employers to ex- 
press their thoughts and ideas on the relevance of 
qualifications in the labour market are a promising 
route to gathering data on VET supply and helping 
complete the feedback loop between labour market 
actors and VET providers. Such surveys, if used in 
a targeted way, can draw a picture of how satisfied 
employers are with the VET graduates they have re- 
cruited and the learning outcomes they bring to the 
workplace. Employers are the best-placed observers 
and finest analysts of the link between intended and 
acquired learning outcomes. 

The Cedefop study developed a prototype of an 
employer reflection survey focused on VET providers: 
it concerned the two occupations which were also 
used for testing the reference points — healthcare as- 
sistant and ICT technician — and was conducted in the 


() This document complements the Europass CV, allowing for a 
more detailed description of the purpose and level of a per- 
son’s qualification, including its learning outcomes, and pro- 
viding information on the education system in which it was 
awarded. 


(®) A technical working group, mandated by the EQF Advisory 
Group, will be set up in September 2021 to follow up on this. 
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form of three different questionnaires addressed to 
VET schools, graduates and employers in two coun- 
tries (Lithuania and the Netherlands). Respondents 
were invited to answer questions relating to a certain 
set of skills (building on the comparison of existing 
reference points) and assess to what extent the VET 
programme had actually provided graduates with 
these skills. 

Most respondents considered the structure and 
level of detail of the description of the skill sets to 
work well. Their answers make it clear how important 
it is for them to have a learning-outcomes-based ref- 
erence point to judge and evaluate graduates’ VET 
skills properly. 


POTENTIAL APPLICATIONS 
The work in this area is part of longer-term Cedefop 
research into the changing content and profile of VET 
qualifications. Recent efforts, such as the analysis 
and comparison of VET curricula under the Future 
of VET project, have yielded promising first insights 
into the feasibility of this undertaking. To pursue this 
strand of work, however, the methods and approach- 
es presented in this summary need to be further de- 
veloped. The current study, laying the groundwork for 
analysing and comparing VET qualifications, high- 
lights both the opportunities and the challenges. 
= Analysis and comparison requires terminological- 
ly and conceptually sound reference points. While 
ESCO boasts a broad portfolio of occupations and 
a multilingual approach, it would need to be sim- 
plified or combined with other reference points for 
the purpose of the employer satisfaction surveys. 
= The upscaling of methods for qualification analysis 
and comparison can be greatly enhanced through 
the wider introduction of qualification databases 
and —- this is paramount - an agreement on, and 
the implementation of, common data formats and 
the development of a common template for pre- 
senting learning outcomes. These steps could be 
decisive in removing obstacles to a (partly) auto- 
mated analysis. 
= The analysis and comparison of VET qualifications 
depend on the quality and coverage of their de- 
scription in terms of learning outcomes. Hence the 
future implementation of this approach requires 
continued, extensive dialogue and sharing of ex- 
periences. While national stakeholders decide how 
to balance and combine the knowledge, skills and 
competences included in a qualification, the way 
these components are described and shared has 
to be based on acommon, agreed template, nota- 
bly for length, structure and terminology. While the 
content of national qualifications will always differ, 
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it must be possible both for learners and employ- 
ers to understand what qualifications are on offer 
in a given EU Member State, and how they com- 
pare to qualifications in another. The quality of the 
learning outcomes descriptions will decide wheth- 
er national authorities can learn from one another 
and, in this way, improve overall national qualifica- 
tions. 


This first exploration of methods to analyse and 
compare qualifications provides a stepping stone 
for future research in a number of areas, notably on 
supporting transparency and recognition of qualifica- 
tions. It will also play a potentially important role in the 
preparation of European vocational core profiles (°), 
as confirmed in 2020 in the recommendation on VET 
and in the joint Osnabriick declaration. 


() Announced in the 2020 European Skills Agenda, page 10, 
footnote 33. 
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